A Task-Oriented Approach for the Development of a Test Collection for Music Information Retrieval
نویسندگان
چکیده
This paper addresses the design of a test collection for CBMR, by applying a task-oriented approach. After having created a test document set, it is proposed to provide assessors with a number of topics and to build musical queries starting from music documents, or documents excerpts, being particularly relevant to the topic. Relevance judgments are then built based on the similarity between this document and the other documents in the collection. Being far to solve all the problematic issues related to the design of a test collection, this paper aims to give some insights and procedures to carry on the test collection design process, which requires many human resources and different types of expertise.
منابع مشابه
Review of ranked-based and unranked-based metrics for determining the effectiveness of search engines
Purpose: Traditionally, there have many metrics for evaluating the search engine, nevertheless various researchers’ proposed new metrics in recent years. Aware of this new metrics is essential to conduct research on evaluation of the search engine field. So, the purpose of this study was to provide an analysis of important and new metrics for evaluating the search engines. Methodology: This is ...
متن کاملPrototyping a Vibrato-Aware Query-By-Humming (QBH) Music Information Retrieval System for Mobile Communication Devices: Case of Chromatic Harmonica
Background and Aim: The current research aims at prototyping query-by-humming music information retrieval systems for smart phones. Methods: This multi-method research follows simulation technique from mixed models of the operations research methodology, and the documentary research method, simultaneously. Two chromatic harmonica albums comprised the research population. To achieve the purpose ...
متن کاملکاربست مدل بازیابی تخصص برای یافتن نویسندگان خبره
This research applied Expertise Retrieval model for finding expert authors, and used evaluation methods of Information Retrieval systems for measuring the performance of those models. Current research is an experimental one. Besides, a variety of methods including survey method has been used in the research process. Various models were developed for finding expert authors, all built on a known ...
متن کاملA User-Oriented Approach to Music Information Retrieval
Search and retrieval of specific musical content such as emotive or sonic features has become an important aspect of Music Information Retrieval system development, but only little research is user-oriented. We summarize results of an elaborate user-study that explores who the users of music information retrieval systems are and what structural descriptions of music best characterize their unde...
متن کاملTowards Large Databases for Music Information Retrieval Systems Development and Evaluation
In the context of MIR/MDL evaluation, a key component for evaluation would be the availability to the research community of a large corpus of test data consisting of both audio and structured music data. This paper proposes a possible path towards this goal by following the basic principles of the SpeechDat projects. SpeechDat refers to successive EC supported projects of large scale multilingu...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2002